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ABSTRACT 

The National Household Education Survey (NHES) was 
conducted for the first time in 1991 as a way to collect data on the 
eariy childhood education experiences of young children and 
participation in adult education. Because the NHES methodology is 
relatively new, field tests were necessary. A large field test of 
approximately 15,000 households was conducted during the fall of 1989 
to examine several methodological issues. This report describes the 
design of the field test and the outcomes of its collection 
activities. Unit and item response rates and the burden associated 
with survey participation are described. The telephone methodology o^ 
the NHES survey offers some important advantages for the program; it 
IS less expensive than a comparable personal interview survey, and it 
can also be instituted relatively quickly, address several subject 
matter areas at the same time, and provide results in a very timely 
fashion. The NHES offers the potential for examining education issues 
that have been difficult to address through the more traditional 
National Center for Education Statistics surveys that are based on 
institutional sampUs. The field test has shown that many of the 
concerns can be handled adequately. Fifteen tables and six figures 
illustrate the overview. (SLD) 
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Foreword 

The National Household Education Survey 
(NHES) represents a major new initiative of the 
National Center for Education Statistics (NCES). 
Befvveen February and May of 1991, the NHES was 
fielded for the first time £s a mechanism for collecting 
data on two different sectors of education policy 
interest: the early childhood education experience of 
young children and participation in adult education. 
Because the NHES methodology is relatively new and 
relies on some innovative approaches, a field test of the 
methodology was an essential first step in the 
development of the survey. Many of the methods of 
evaluated during the 1989 NHES field test were adopted 
for the full-scale survey. 

A large field test of approximately 15,000 
households was conducted during the fall of 1989. A 
number of methodological issues associated with 
collecting and analy2dng data on education issues from 
a random digit dialing telephone survey were examined. 
This report is one of five that describe the 1989 NHES 
Field Test experience. The five reports are the first in 
a series of technical publications pertaining to the 
design and conduct of the NHES that NCES hopes to 
continue in the years to come. NCES believes that the 
reports contained in this series will provide users of the 
NHES data with a better understanding of the NHES 
methodology and that they will assist the survey design 
efforts of others. 

The first report in this series. Overview of the 
National Household Education Survey Field Test, 
describes the design of the field test and the outcomes 
of the field test data collection activities. It reports on 
the response rates obtained, both unit and item, and the 
burden associated with survey participation. Each of 
the next four reports in the series focuses on a specific 
issue that was examined in the 1989 NHES field test. 



The second report. Telephone Under coverage 
Bias of 14- to 21 -Year-Olds and 3- to 5-Year-Olds, 
analyzes data from the Current Population Survey to 
identify the extent of telephone coverage for two 
distinct populations of interest and the bias associated 
with this type of undercoverage for estimates of school 
dropouts and early childhood education program 
participation. Methods for adjusting survey estimates 
to partially reduce this bias are developed and 
evaluated. 

The third report, Multiplicity Sampling for 
Dropouts in the NHES Field Testy examines a technique 
that was used to increase the coverage of 14- to 21- 
year-olds and to capture more dropouts Ln the sample. 
The report describes the effectiveness of the multiplicity 
sample in achieving these goals. 

The fourth report. Proxy Reporting of Dropout 
Status in the NHES Field Test, focuses on measurement 
errors arising from the use of proxy respondents. 
During the 1989 Field Test, a knowledgeable household 
member was used as a source of information on the 
school enrollment of each sampled 14- to 21-year-old in 
the household. In addition, 14- to 21-yeai-olds were 
asked to report on their own school enrollment. The 
report describes the correspondence between the 
responses given by proxy respondents with those 
provided by the youths themselves. 

The fifth report. Effectiveness of Oversampling 
Blacks and Hispanics in the NHES Field Test, describes 
the approach used to increase the number of black and 
Hispanic households/youth in the sample. During the 
field test, an approach that uses demographic 
information at the telephone exchange level to develop 
sampling strata was used to oversample black and 
Hispanic households. The report examines the yield of 
the field test santple design versus that which would 
have been expected without oversampling. The effects 
of oversampling on the precision of survey estimates 
are reported. 
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Introduction 

The National Household Education Survey 
(NHES) is the first major attempt by the National 
Center for Education Statistics (NCES) to go beyond its 
traditional, school-based data collection systems to a 
household survey. A household survey has the potential 
of providing the types of data needed to address many 
current issues in education. Included among issues that 
can be assessed through surveys ot persons in their 
households are dropping out of school, preschool 
education, access to postsecondary , adult and continuing 
education, the status of former teachers, extra-school 
learning, and home-based education. 

NCES has a legislative mandate to collect and 
report information on the condition of education in the 
United SUtes. To fulfill this mandate, NCES 
historically has sampled educational institutions to 
collect data about teachers, students, schools, school 
districts, and state education agencies. The collection 
of data from noninstitutional samples of individuals, 
particularly through household-based surveys, has been 
limited. The NHES represents a new initiative for 
NCES, and holds the promise of greatly enhancing the 
scope of the issues covered by the data collection 
activities of the Center. 

The NHES is a mechanism for collecting 
detailed information on educational issues from a 
relatively large and targeted sample of households in a 
timely fashion. It fills a need that existing household 
surveys, such as the Current Population Survey (CPS) 
and the Survey of Income and Program Participation 
(SIPP), cannot satisfy because they are designed to 
focus on issues other than education. Consequently, 
when these existing surveys are used to collect data on 
topical educational issues, it must be done in a limited 
and supplementary manner. The level of detail in such 
supplements is limited because the respondent is already 
burdened by the basic (or core) survey data items. In 
addition, researchers who use a supplement to an 
existing survey to collect data on other issues lack the 
ability to target the sample to particular population 
groups because the sample designs were established to 
meet other objectives. 

The NHES Field Test was targeted to the 
groups of interest by screening and sampling devices. 
The instruments were designed to address the selected 
issues in detail so that analyses could be performed to 
help illuminate the phenomena of interest. 



Furthermore, the data collection methodology was 
specifically designed so that relatively complex issues, 
such as the preschool arrangements of children, could 
be handled snaoothly and efficiently. 

This report provides an overview of the design 
and procedures for the NHES Field Test, which was 
conducted in the Fall of 1989. In addition, this report 
provides a detailed analysis of survey response rates, 
for both the instruments and individual items, and an 
analysis of the administration times of the various Field 
Test instruments. 



Survey Topics 

NCES has as its legislative mission the 
collection and publication of data on the condition of 
educall )n in the Nation. The National Household 
Education Survey is specifically designed to support this 
mission by providing a means to address educational 
issues that caimot be adequately studied through the 
Center's traditional, institution-based data collection 
efforts. 

As a first step in reaUzing the potential of 
NHES, two topics of broad policy interest were selected 
for the Field Test. One topic concerned dropouts. The 
1988 Amendments to the School Improvement Act 
(P.L. 100-297, Sec. 406) require NCES to: 

M (g)(4)(A) \ . .conduct an annual survey of 
dropout and retention rates as an education 
indicator. . 

In response to this mandate, NCES is in the 
process of developing a dropout statistics program. It 
is examining the feasibility of several different 
approaches to collecting annual data on dropouts. The 
NHES is one possible component of this program. 

The second topic covered by the Field Test 
was the eariy childhood educational experience of 3- to 
5-year-old children. As more women enter the labor 
force and as children are exposed to a greater variety of 
out-of-home experiences prior to first grade, educators, 
policymakers, and researchers have become 
increasingly interested in the early educational 
experiences of preschool children. The national 
education goal pertaining to school readiness has 
focused even more attention on early life experiences 
and the role they play in preparing children for 
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successful school experiences. However, limited 
information is available at the national level that may be 
used to monitor these experiences. 

The early educational experiences of young 
children cannot be fully addressed through facility-bas-id 
studies because so many preschool-aged children are 
cared for only in their own or otlier private homes. In 
addition, there is no comprehensive list of all child care 
facilities end preschools in the United States from 
which a sample can be drawn, and many places 
providing child care are not formally licensed or 
identified on official lists. The data collected in the 
NHES Field Test sample can be used to help direct 
NCES projects aimed at improving the information base 
on younger children, and may provide information that 
will be useful for the design of future longitudinal 
surveys of preschool children. 



Study Components 

The National Household Education Survey 
Field Test consisted of two general components. The 
first was the telephone survey component, which 
addressed school dropouts and early childhood 
education. The second was a school-based approach 
designed to test an alternate methodology for estimating 
dropout rates.' This report focuses on the telephone 
survey component. In general, when this report refers 
to the •*NHES,'* it is discussing the telephone sur\'ey 
component of the Field Test, unless the school-based 
component is specifically mentioned. 



Field Test Telephone Survey 

The NHES methodology is relatively new, and 
some of the approaches are innovative. For this 
reason, a Field Test of the methodology was an 
essential first step in the development of an approach 
for an ongoing household telephone survey effort. In 
particular, the Field Test addressed the methodological 
issues associated with collecting and analyzing data on 
educational issues from a telephone survey. 

The Field Test can be viewed as a small-scale 
NHES in which a smaller number of households were 
sampled than would be included in the full-scale survey. 
Other than the size of the sample, and the consequent 
reduction in analytic capability, the Field Test was 
intended to use all of the features of the full-scale 



NHES surveys, including the use of random-digit- 
dialing (RDD) and computer assisted telephone 
interviewing (CATI) methods. 

Four interviews were conducted during the 
Field Test in order to address the two survey topics of 
interest: 

■ A Skreener Interview, which was used to 
enumerate household members, determine 
eligibility to proceed with other interviews, 
and collect basic household characteristics, 

" A Houseliold Respondent Interview (HRI), 
which collected information on the educational 
status of 14- to 21-year-olds from a 
knowledgeable adult household member; 

■ A Youth Interview, which collected data on 
educational status, experiences, and 
expectations from sampled 14- to 21-year-old 
youth; and 

■ A Parent/Guardian Interview, which 
collected information on the child care and 
preschool arrangements for 3- to 5-year-olds, 
as well as information on the educational 
environment in the home. 



Flow of NHES Field Test Interviews 

Figure 1 shows the flow of the NHES 
interviews. The process began with the Screener, in 
which questions were asked about the household and its 
phone number(s), the eligibility of the household (any 
members under 65) was determined, and the household 
members enumerated. Information on household 
characteristics was also collected in the Screener. 

If the household contained any persons ages 14 
to 21 years, the Household Respondent Interview (HRI) 
was used to collect information about the youths' 
educational statuses for sampling purposes. In addition, 
if there was a sampled woman 28 to 64 years of age, 
information was collected about the educational status 
of any of her 14- to 21-year-old children who live 
outside the household for the multiplicity sample 
(discussed below). 

The results of the Screener and the HRI 
determined the extended interviews to be conducted, if 
any. Parent/Guardian Interviews were conducted in any 
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Figure 1. - Flow of NHES Interviews 
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household in which there were 3- to 5-year-old 
children. Youth Interviews were conducted with all 
youth who met die study definition of a dropout, and 
with a sample of non -dropout youth. Since some of the 
youths were qualified to complete the HRl (those 18 
years or older), the HRl airid Youth respondents were 
occasionally the same person. 

Many households required more than one 
extended interview to be conducted with various 
household members. In some cases, the same person 
was the Screener respondent, the Household 
Respondent, a sampled youth, and the Parent/Guardian 
of a 3- lo 5-year-old. 

Every effort was made to complete all 
applicable interviews with the respondent who answered 
the Screener at the time the Screener was completed. 
Interviewers then asked for any other household 
members to be interviewed. 

The sections below describe the content of the 
extended interviews. Flowcharts depicting the extended 
interviews appear in section 4 of this report. 



Parent/Guardian Interview 

The Parent/Guardian Interview was designed 
to collect information on the early childhood educational 
experiences of 3- to 5-year-olds. This interview was 
conducted with the parent or guardian who said he or 
she knew the most about the child's care and education. 
Typically, in about 75 percent of cases, the respondent 
was the child's mother. In about 20 percent of cases, 
it was the father who responded to the interview. 
About 5 percent of respondents were other persons such 
as a grandparent, a foster parent, or other guardian. 

The interview was composed of 5 general 
sections. The first section established the eligibility of 
the child by confirming the month and year of birth, 
and established the relationships of all household 
members to the child. 

The second section of the interview concerned 
only 4- and 5-year-olds. The questions asked about 
enrollment in kindergarten or first grade, and went on 
to ask about specific aspects of the child's school. The 
third section identified arrangements in which children 
spent time and a series of questions was asked about 
each child c^re or preschool arrangement. This series 
of questions collected information on the setting, care 



provider, cost, hours, child-staff ratio, educational 
content, activities, reasons for selecting this care, and 
the parents' activities while the child was in care. 

The fourth sc jiion concerned any hajidicapping 
condition the child may have had, and any handicapped 
services the child received. The final section concerned 
the child's educational environment at home. We 
discussed with the parent the materials available to the 
child, reading activity with the child, television 
viewing, and the parents' own educational levels. 

Household Respondent Interview (HRl) 

The Household Respondent was a 
knowledgeable adult member of the household who 
reported on the education status of the household youth. 
In the case of the multiplicity sample households 
(discussed below), HRIs were conducted for the out-of- 
househoJd youth of sampled women. The HRl serves 
three important ftmctions in NHES. 

■ First, it collects important information for al! 
14- to 21 -year-olds without having to interview 
each separately. These data can be used to 
develop national estimates of numbers of 
dropouts and dropout rates. 

■ Second, the HRl provides the information 
required for sampling 14- to 21 -year-olds for 
the Youth Interview 

■ Finally, by comparing the responses of house- 
hold respondents and youths, it is possible to 
assess similarities and differences in reports by 
youth themselves or by household proxy 
respondents. 

The HRl began with the current enrollment 
sUtus of the youth (i.e., whether the youth was enrolled 
in school during the last week, whether the youth was 
attending on a regular basis, and the type of program 
attended). The youth's enrollment status 1 year ago 
was then determined using parallel items. For those not 
enrolled in elementary/secondary school (el/sec), the 
diploma status was determined as well. These data 
served as the basis for sampling for the Youth 
Interview. 

The HRl also collected information on whether 
the youth had any handicapping condition and whether 
the youth had received special education services, both 
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currently and in the past. The next section of the 
interview was concerned with employment status: 
whether the youth was employed, how many hours per 
week, at what wage, and whether the youth had been 
seeking work. Finally, the HRI collected background 
information such as the parents' educational statuses. 

Overview of the Youth Interview 

The Youth Interview was conducted with each 
sampled youth after the HRI was completed. Selection 
of youth for the extended interview was determined 
from enrollment, attendance, and diploma status items 
reported in the HRI. 

The Youth Interview contained almost all the 
items from the HRI, permitting comparison of youth 
responses with those of the household respondent. The 
interview began by collecting current enrollment status 
of the youth (i.e., whether the youth was enrolled in 
school during the week prior to the interview, whether 
the youth was attending on a regular basis, and whether 
youth was attending any out of school program). The 
youth's enrollment status one year ago was then 
determined using parallel items. For those not 
enrolled, diploma status was obtained. 

Youth identified as dropouts were asked a 
series of questions about leaving school and their plans 
for re'nming and becoming certified. All youth were 
asked questions about educational experiences while 
enrolled in elementary /secondary school, including 
services for handicapping conditions. The final series 
of questions collected infoniiation on employment status 
and family background. 

M^or Methodological Issues 

The NHES Field Test included a number of 
tests and evaluations of issues related to the design and 
conduct of a telephone survey of education. Findings 
from these tests required consideration before a full- 
scale system could be established. These issues, briefly 
reviewed here, are discussed in detail in later reports in 
this series. 

■ Oversampling: The feasibility of 
oversampling blacks and Hispanics to improve 
the sample yield and estimates for these 
subgroups was evaluated in the Field Test. 
This was accomplished by increasing the 



probability of selection for those telephone 
clusters that were identified as containing 20 
percent or more black or Hispanic residents. 
The results of this test appear in the report, 
Effectiveness of Oversampling Blacks and 
Hispanics in the NHES Field Test. 

« Proxy Reports: Another major methodo- 
logical issue examined in the Field Test is the 
correspondence between the responses of 
knowledgeable household respondents and 
those of the 14- to 21 -year-old youths 
themselves about their educational status. In 
many surveys, one person responds to 
questions for all the members of the 
household. The responses of this household 
respondent are then used to classify the 
ind;vidualsby reported characteristics, such as 
dropou* status. However, the responses of the 
youths themselves may differ from those of the 
respondent for a number of reasons, including 
perspectives and different knowledge. In the 
Field Test, both the household respondent and 
the youth were asked similar questions and the 
responses of the two sources were compared. 
The results of this comparison, which have 
implications for the use of single household 
respondents, appear in the report. Proxy 
Reporting of Dropout Status in the NHES Field 
Test, 

■ Population Coverage: One of the major 
concerns in any survey, especially a household 
survey conducted via telephone, is the failure 
to "cover" all persons in the population. In 
other words , there is concern that some 
persons who are eligible to be in the sample 
are excluded for a variety of reasons. The 
coverage issue, including the much publicized 
problems concerning some minority group 
members in the decennial census, is a major 
concern for all household surveys. In addition 
to the typical household coverage problems, a 
telephone survey fails to include about 7 
percent of the U.S. population who live in 
nontelephone households.^ 

An important issue associated with the 
telephone coverage problem is the concern that 
those who live in nontelephone households 
may have different characteristics than those 
who live in households with telephones. If 
these differences are large and data from 



telephone households are used to estimate 
characteristics of the entire population, the 
resulting biases may have important 
consequences. A detailed analysis of 
telephone imdercoverage bias was undertaken 
for the NHES. The results of this analysis and 
a discussion of special estimation techniques to 
reduce the effects of undercoverage bias 
appear in Undercoverage Bias in the Field Test 
for the National Household Education Survey, 

■ Multiplicity Sampiiing: The multiplicity 
sampling approach is one of the techniques that 
was tested specifically because of the concern 
about the coverage problem. In the Field 
Test, a 14- to 21-year-old youth was eligible 
for the sample not only if his or her household 
was sampled, but also if the household of the 
youth's mother was sampled. This approach 
was tested to see if it improved the coverage 
of the 14- to 21-year-old population and if it 
resulted in the capture of more dropouts 
through the sample of telephone numbers. An 
analysis of the effects of multiplicity sampling 
is presented in Multiplicity Sampling for 
Dropouts in the NHES Field Test, 

Another issue that arose in the NHES Field 
Test is the complexity of the data collection and how 
well this can be addressed in a telephone survey. The 
diversity of child care and preschool arrangements and 
the presence of multiple children in a household was 
ased to inv^ tigate the methodological capability of the 
NHES to address complex education-related topics. In 
addition, the NHES Field Test design called for 
multiple interviews per household. For example, a 
household might include one or more 3- to 5-year-old 
children and one or more 14- to 21 -year-old youths. 
The need to collect information on each of these 
persons led to multiple interviews per household. This 
increased burden could have resulted in serious 
problems in response rates, and required careful 
assessment in the Field Test. 



Telephone Survey Sample Design 

The primary objective of the Field Test of the 
NHES was to investigate important methodological 
issues that are critical to evaluating the feasibility of 
using a telephone survey to collect data on education 
topics. The best mechanism for evaluating these issues 
was to conduct a survey under conditions as close as 
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possible to those encountered in a full scale survey. 
Thus, the Field Test was conducted as would be any 
other survey, except that it was smaller in size thpn 
anticipated for future studies. 

The two survey components of the Field Test 
were the educational status of 14- tc 21-year-olds, with 
a special emphasis on dropouts, and the early childhood 
education characteristics of 3- to 5-year-olds. As 
noted above, several methodological issues were 
addressed within these surveys. One issue was the 
ability to improve the precision of estimates for black 
and Hispanic persons by oversampling methods. Other 
issues included the use of an extended definition of 
household membership to improve coverage of 14- to 
21-year-olds, the "sharing" of the cost of screening 
households by using the households for more than one 
survey, and the study of the response characteristics of 
proxy respondents. 

Population Sampled 

The NHES Field Test sample was dravm from 
the telephone households of the noninstitutionalized 
civilian population of the 50 states and the District of 
Columbia. Telephone households include about 93 
percent of the U.S. noninstitutionalized civilian 
population. The sample was designed to obtain 
information for two age groups: 3- to 5-year-olds and 
14- to 21-year-olds. The approximate numbers of 
persons in these populations (based upon 1988 CPS 
estimates) are shown in the following table. 

Populations Population Size 

Persons in households 
3- to 5-year-olds 10,949,000 
14- to 21 -year-olds 27,983,000 

Persons in telephone households 
3- to 5-year-olds 9,684,000 
14- to 21 -year-olds 25,682.000 

In addition to the restriction of the sample to 
persons living in telephone households, the NHES 
sample excluded members of the Armed Forces and 
institutional populations (e.g., inmates, patients in long- 
term care facilities). This restriction had a very small 
impact on the sample of households with 3- to 5-year- 
olds, since these persons are not members of the 
excluded populations, except in very rare cases. 
However, the impact on the 14- to 21-year-old 
population could be more significant. 



The Census Bureau has the same type of 
restriction in its CPS and the impact of the restriction 
was recently studied with the following results. 

The yvrmed Forces and the institutional 
population differ greatly from the total 
population in age-sex structure. . . On March 
1, 1988, males 18 to 64 years old constituted 
90.0 percent of the Armed Forces population 
as compared with 30.5 percent of the total 
population, and females 65 years and over 
constituted 41.8 percent of the institutional 
population as compared with 7.3 percent of the 
total population. However, these two groups 
together accoimted for only 2.1 percent of the 
of the total population, and as a result, the 
civilian noninstitutional population (which 
accoimted for 97.9 percent of the total) had an 
age-sex structure very similar to the total 
population. Similarly, the social and economic 
characteristics of the Armed Forces and of the 
institutional population could differ greatly 
from those of the total population with 
relatively small differences between the 
characteristics of the total population and of 
the civilian noninstitutional population.^ 

The excluded population (institutionalized and 
noncivilian) is heavily concentrated in the over-21 
segment of the population. As a result, the bias 
resulting from their exclusion was assumed to be 
relatively small for the NHES survey topics. The bias 
is small even if the difference between the excluded 
population and the sampled population is large, because 
the percentages of 3- to 5-year-olds and 14- to 21-year- 
olds that are excluded are small. 



Random-Digit-Dialing Sample Selection 

The sampling method used for the NHES is a 
variant of procedures developed by Warren Mitofsky 
and Joseph Waksberg.** The Mitofsky-Waksberg 
method produces an equal probability sample of 
households with telephones and requires a smaller 
numbe- of telephone calls than the sampling procedures 
previously used for random digit dialing (RDD). A 
time-saving variant of this method, sometimes referred 
to as the "modified Waksberg procedure," was used for 
the NHES survey. The description below applies to 
both procedures, except the second stage steps which 
differ. 



A list of all existing telephone area codes and 
existing prefix number within the areas was determined 
based on data tapes provided by AT&T. All possible 
combinations ^or the next two digits were added to the 
set of prefix areas. Thus, a list was established of all 
the possible first 8 digits of the 10 digits in telephone 
numbers. These eight-digit numbers were treated as 
Primary Sampling Units (PSUs). 

A random selection was made of an eight-digit 
number from this list, and a two-digit random number 
was added to create a full telephone aum'oer. The 
number was dialed to determine if it was residential. If 
it was residwitial, the PSU was retained in the sample. 
If the dialed number was not residential, the PSU was 
rejected and no further calls within the PSU were 
made. Additional residential PSUs were selected in the 
same way until 1,000 were chosen. In the original 
Mitofsky-Waksberg method, within each residential 
PSU, the final two-digit combinations are selected at 
random and dialed within the same eight-digit group 
until a specified number of completed interviews is 
obtained. This sampling procedure produces an equal 
probability sample of telephone households. 

The reason for sampling clusters of telephone 
numbers (the eight-digit PSUs), and then households 
within those clusters, rather than taking a simple 
random sample of telephone numbers is cost savings. 
The cost of the cluster sample is considerably below 
that of simple random sampling because there is 
considerable savings in the telephone operations. We 
estimate that with cluster sampling, the reduction in the 
number of telephone numbers to be dialed is at least 
50 percent. The price paid for the cost savings is a 
small reduction in the precision of the estimates. 

The procedure described above is a sequential 
process. It is not possible to determine in advance how 
many telephone numbers in a cluster need to be dialed 
in order to achieve the desired sample size of 
households. This is awkward to implement when a 
survey faces a tight deadline, as was the case with the 
NHES Field Test. An alternative, and faster, sampling 
method is to use a fixed number of telephone numbers 
per cluster, rather than a fixed number of households. 
This also provides a probability sample, but it is no 
longer self-weighting. The weight in each cluster is 
proportionate to the reciprocal of the number of 
sampled households in the cluster. This is the 
"modified Waksberg procedure" that was used for the 
NHES Field Test. This variant is described in more 
detail by Brick and Waksberg.^ 
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Sample Sazes Required to Achieve Target Numbers 



Oversampling Blacks and Hispanics 



The Field Test sample wa5 designed to identify 
4,000 households with either a 3- to 5-year-old or a 14- 
to 21 -year-old. Using the October 1986 CPS data tape, 
estimates were that about 15,000 households had to be 
screened to meet these goals. 

The number of telephone numbers that must be 
dialed in order to obtain a sample of 15,000 households 
can be estimated in two steps. The first step is to 
identify household clusters. A sample of about 5,000 
telephone numbers was needed to identify the 1,000 
clusters since it takes an average of five telephone 
numbers to find one residential cluster. The second 
step is to identify residential phone numbers within the 
retained residential clusters. Typically, it takes about 
five telephone numbers to identify three residences once 
the cluster is identified as residential In addition to the 
60 percent residency rate, an 80 percent cooperation 
rate was assumed for the survey. Therefore, in order 
to achieve a sample of 15,000 participating households, 
30,000 telephone numbers (30,000 x .60 x .80 is 
approximately equal to 15,000) were required. In each 
retained sampled cluster, 30 random numbers were 
selected. 

The final step in the process was the sampling 
of individuals from the households. Ehiimates from the 
1986 CPS indicated that the mean number of 14- to 21- 
year-old members was 1.4 in households with 14- to 
21-year-olds. The mean number of 3- to 5-year-olds 
was 1.1 in households with 3- to 5-year-olds. These 
means do not vary much across race or ethnicity. 

All identified 3- to 5-year-olds were included 
in the sample, but a further stage of sampling was 
introduced for the 14- to 21 -year-olds. A Household 
Respondent Interviw (HRI) was conducted for each 14- 
to 21 -year-old. All 14- to 21 -year-olds who were 
identified from information supplied by adult household 
respondents as being dropouts or not enrolled in school 
were sampled for an extended interview. All of these 
youth were included because of the need to obtain more 
detailed data on their experiences.^ Only about one in 
five of the remaining youth were sampled for extended 
interviews because this number would be sufficient for 
comparison purposes and it would decrease interviewing 
costs. 



Oversampling is generally introduced in 
surveys for one of two reasons. The first is to improve 
estimates of totals for certain domains. For example, 
if a substantial portion of a target population occurs in 
a small and well-defined subset of the total population, 
oversampling this subset reduces screening costs for a 
desired sample size. The second is to increase the 
sample size for domains of special analytic interest, for 
which an equal probability sample would not provide a 
sufficient sample. In the NHES, oversampling was 
studied for the latter reason. 

Oversanqjling to increase sample sizes for 
blacks and Hispanics was used in the Field Test. 
Oversampling in a random-digit-dialing survey is 
typically limited because of the lack of information 
associating telephone numbers with the groups to be 
oversampled and the relatively high cost associated with 
screening households. For the Field Test, information 
from a Donnelley Marketing Information Services 
computer tape was used to oversample both black and 
Hispanic households The tape contained 1980 Census 
characteristics by telephone exchange (the area code and 
first three digits of the telephone number) and these 
were used to stratify the population of telephone 
exchanges by percenUge black and Hispanic. 

Telephone exchanges with concentrations of 20 
percent or more blacks and Hispanics were identified 
and oversampled at a rate of 2 to 1. Note that the 
oversampling was within exchanges rather than 
restricted to black and Hispanic households. It would 
have been possible to subsample nonblack and non~ 
Hispanic households in high minority exchanges areas 
to create a self-weighting sample for persons who are 
not black or Hispanic. However, because of the 
extensive screening required to locate members of any 
rare population, such as dropouts, it is clear that this 
loss in sample size would have been quite inefficient. 
A detailed analysis of the sample yield and the effects 
on the variances for these subgroups is presented in the 
report, Oversampling Blacks and Hispanics in the 
NHES Field Test. 



Extended Household DeHnitions 

One method for improving the coverage of a 
household survey is to consider alternative definitions 
of household residency. The typical household 
definition only mcludes persons who are currently 
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living in the household. For the 14- to 21-year-old 
component of the NHES Field Test, an extended 
definition of household residency was used. College 
housing residents, age 21 or less, were treated as 
members of their parents* households, and were given 
a chance of selection in the same way as other 
household members. When such a college housing 
resident was sampled for a Youth Interview from a 
household roster, the parent (or other respondent to the 
screening) was asked for a telephone number at which 
the student could be reached. These college students 
were sampled only as members of their parents' 
household. 

A further extension of the definition of persons 
in a household involved multiplicity sampling. In the 
Field Test, the multiplicity sampling approach was used 
for approximately one-fourth of the households. For 
this subsample, all women aged 28 to 65 in the 
household were asked to provide information about 
their children who were 14 to 21 years old and n^t 
livmg in their household. The youth identified in this 
process were included in the Field Test sample. 
Telephone numbers and addresses for nonhousehold 
youth sampled for Youth Interviews were requested and 
used to trace the youth for interviewing. 

There were two major reasons for utilizing 
multiplicity sampling. First, the coverage of dropouts 
in households with telephones and in general household 
surveys was a major concern. By extending the 
number of persons who are identified as being 
associated with a household in this fashion, it was 
expected that the sample would include some 14- to 21- 
year-olds from nontelephone households, who otherwise 
would not have had a chance of being included in the 
sample. In other words, there was some potential to 
reduce the coverage bias by using this method. 

A second reason for studying this approach 
was that it provided a method for locating youth, 
especially dropouts, at a much reduced cost. With 
relatively little expense, the locating information could 
be used to include those youth who were identified as 
having a higher probability of being dropouts. A 
detailed analysis of the effectiveness of the multiplicity 
sampling procedure is presented in the report. 
Multiplicity Sampling for Dropouts in the NHES Field 
Test. 



Procedures for Cooducting Interviews 

The following sections discuss the proceiures 
used in the daU collection phase of the NHES Field 
Test, including the use of computer assisted telephone 
interviewing (CATI), staff training, interviewer 
assignments and contact procedures, and quality 
control. 



CATI System Advantages 

The use of a CATI system for the NHES 
offered a number of advantages that facilitated the 
implementation of NHES survey procedures. Briefly, 
the most salient features of the CATI system for NHES 
were as follows: 



n Project Administration: The CATI system 
reduced substantially the amount of paper 
recordkeeping ordinarily required for a 
telephone survey, such as contact records, and 
provided up-to-date reports on the status of 
cases, numbers and types of contacts, sizes of 
standing work queues, and interviewer 
productivity and response rates. 

■ Eligibility and Sampling: CATI 
automatically identified household and persons 
eligible for the NHES survey topics. It also 
selected appropriate households for the detailed 
household respondent interview, sampled 
households for multiplicity sampling questions, 
and sampled respondents for extended 
interviews. 

■ Scheduling: CATI fed telephone numbers to 
the interviewers, maintained a schedule of 
callback appointments, and rescheduled 
unsuccessful contact attempts to the 
appropriate day and time according to the 
protocol for the study. 

■ Skip Patterns: The CATI system 
automatically guided interviewers through the 
complex skip patten in the questionnaire, 
reducing the potential for interviewer error. 
This is particularly important for a survey in 
which repeated series of questions are asked 
about a particular subject, such as child care. 



■ Receipt Control: The CATI system provided 
automatic receipt control in a flexible manner 
which was used to produce a variety of status 
reports. 

The use of CATI for the NHES Field Test was 
critical because of the difficult skip patterns that arise 
when dealing with complex issues such as those 
associated with the educational activities of 
preschoolers. Without CATI, this would be a very 
difficult survey to administer, especially by telephone. 
The use of CATI is also very helpful when more than 
one topic is covered in a survey system, and when 
sampling of households or household members is 
required based on individual or household information 
collected during the survey, as it was in the NHES 
tield Test. 



Interviewer Training 

Interviewer training was conducted during the 
last week of September and the first week of October 
1989. Interviewers were trained in two groups of about 
28 each. 

A total of 20 hours of project-specific training 
was provided to each interviewer in addition to the 
basic traijiing in general intei-viewing techniques and the 
use of the CATI system. The NHES interviewer 
training was conducted during three four-hour week 
night sessions and one eight-hour weekend session for 
each group. 

Interviewer training was conducted using the 
CATI system throughout. In this way, the trainees 
actually entered information in the CATI system during 
training scripts and lectures, providing them with 
"hands-on" experience prior to beginning data 
collection. 

The topics covered in the training session 
included an introduction to the study, interactive 
lectures concerning each of the survey questionnaires, 
survey procedures, refusal avoidance, and practice role- 
play scripts. 

Interyiewing Procedures 

Because the sample spanned four time zones, 
interviewing was scheduled to maximize use of evening 
and weekend hours. In household telephone surveys. 
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evening and weekend calls are the most productive 
because adult household members are most likely to be 
home at these times. Conversely, vveekdays and Friday 
and Saturday evenings tend to be less productive, since 
adult household members are less likely to be available. 

NHES interviewers made at least seven 
attempts to screen households in order to determine the 
presence of eligible household members (those aged 3 
to 5 or 14 to 21). These seven calls were 
staggered— made on different days of the week and at 
different times of the day over a 2-week period, and 
included two day calls, three evening calls and two 
weekend calls. Once a household was identified as 
eligible for an extended interview (youth or 
parent/guardian), the maximum number of calls was 
doubled to 14. When repeated attempts to contact the 
respondent proved unsuccessful, the case was referred 
to a telephone supervisor to discuss appropriate methods 
of completing an interview (e.g., holding a case for 
some time and releasing it for additional atterqpts later 
in the data collection period). 

The CATI system scheduled cases 
automatically, based on an algorithm that was 
customized for the NHES sur/ey. CATI assigned cases 
in the following order of priority: 



■ Cases that had specific appointments; 

■ Cases that had unspecified appointment/general 
callback times; 

■ Cases that were busy signals came up 15 
minutes later for another attempt; 

■ Cases that had been attempted previously with 
no contact were tried during other specific 
time frames; and 

■ Cases that were new and had never been 
worked. 

In random^ligitKlial surveys, some clusters of 
telephone numbers yield the expected number of 
households, whereas other clusters contain relatively 
fewer households, requiring that additional numbers in 
these clusters be used in the survey. Those in this 
latter group are referred to as "low-yield" clusters and 
have proportionally fewer households than other 
clusters. In order to quickly identify these low-yield 
clusters, new work was distributed soncwliat more 



quickly in NHES than in other (e.g., list sample) 
telephone surveys. As soon as such a cluster v is 
identified, additional numbers were released in these 
low-yield clusters, to increase the number of households 
contacted. The purpose of increasing the number of 
households in low-yield clusters is to reduce the 
variability in the weights. During the conduct of the 
Field Test, about 590 additional numbers were included 
in the sample. 

Interviewers coded a case as a "language 
problem" when they encountered a non-English 
speaking respondent and there was no evidence of an 
English-speaking household member. An interview was 
conducted in Spanish if that was the respondent's 
language; all other languages received a final 
disposition of "language problem." A hard copy 
Spanish language translation of the survey instruments 
was provided to each Spanish-speaking interviewer, and 
responses were entered into the CATI system as the 
questions were asked. 



Data Collection Quality Control 

Data quality control efforts for the data 
collection phase began during the CATI development 
period. As the CATI system was programmed, 
extensive testing of the system was conducted. This 
testing included review by project research staff, 
telephone interviewing center staff, programmers 
themselves, and data preparation staff. The testing by 
staff members representing different aspects of the 
project was designed to ensure that the system was 
working property from all of these perspectives. 

To "shake down" the CATI system as a further 
effort to be sure that the system was working properly, 
a live pretest was conducted in early September, with 
a small sample of telephone numbers. Actual 
respondents tend to be less predictable and more varied 
m their responses than staff members who are testing a 
system. Therefore, by testing the system in this way, 
staff can determine if the system is ready for use with 
the actual sample. 

Quality control activities continued during 
training and data collection. During interviewer 
training, interviewers were paired with one another and 
conducted role-play interviews on the telephone. 
Scripts for the role plays were developed by project 
staff to include a wide variety of scenarios in terms of 
household composition, interviews to be conducted, and 



response patterns. The interviewers in training did not 
see these scripts in advance. Project staff monitored 
the role plays and provided feedback and additional 
training as necessary. 

When interviewers began actual data collection, 
they were monitorect on an ongoing basis by telephone 
center supervisors. In addition, project research staff 
monitored ih**. mterviewers occasionally. When 
problems were encountered, additional training was 
provided. Data preparation staff also reviewed the 
cases from the CATI system as they were completed, 
and referred problems to the project staff for resolution. 
Interviewer memos were posted and distributed when 
any recurring problems were identified. 

At least once each week, the CAT! 
management system produced computer generated 
reports that displayed response rates, refiisal rates, and 
refiisal conversion rates for each NHES interviewer. 
These reports assisted telephone center supervisors in 
identifying interviewer performance problems that may 
not have been detected through monitoring. 

In addition, daily CATI reports were produced 
that showed the cunent status of all interviews and 
cases. Daily reports were monitored by project staff 
throughout the data collection period, and were 
provided to NCES on a weekly basis. 



Survey Response 

This section describes the outcomes of the 
random-digit-dial NHES Field Test suirey, including 
the unit and item response rates. Because the NHES 
Field Test was designed to examine the NHES as a 
methodological approach to addressing issues in 
education, an important aspect of the methodological 
assessment is the response rates that can be obtained for 
the surveys. Screening response rates are always of 
concern in random-digit-dial surveys, and this is 
particulariy true when fiill household enumerations are 
required. A detailed analysis of unit and item response 
was deemed necessary to the assessment of the NHES 
Field Test. 

Unit Response Analysis 

Unit response rates are defined as the number 
of units (telephone numbers, households, or persons) 
that completed the required interview divided by the 
number of units that were sampled and eligible to 
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complete the interview. In some cases, these rates 
were easily defined and computed. In other cases, the 
rates had to be estimated. The methodologies used for 
estimating the response or completion rates at each 
stage of the interview process are described in this 
section. 

Sampling Clusters and Residential Cluster 
Identification 

As described in the previous chapter, the 
NHES Field Test used a modified Waksberg procedure 
for selecting the sample. The first step in sampling was 
to determine which clusters of telephone numbers would 
be used. For the NHES Field Test, a cluster was 
defined as the first eight digits of a telephone number. 
A random two-digit number was added to the cluster 
number to form a complete telephone number. This 
number is referred to as the prime number. 

A sample of 1,000 residential clusters was 
required for NHES. Because of procedures designed to 
oversample black and Hispanic households (using the 
oversampling procedure described by Mohadjer'O, two 
sets of clusters were sampled. One set was for areas 
with more than 20 percent minority residence and the 
other for all other areas. The rationale for 
oversampling these areas is described in detail by 
Mohadjer^. The basic intent is to sample areas with 
high concentrations of blacks and Hispanics without 
seriously affecting the precision of totals across all 
areas. 

A total of 337 high minority-concentration 
clusters and 663 low minority clusters were targeted. 
To obtain them, a sample of 5,000 clusters was 
generated and telephoned to identify residential clusters. 
Typically, 20 to 25 percent of clusters are retained 
using these procedures. For the Field Test, 25 percent 
of the clusters were retained. Tliis total included 31 
percent of the high minority-concentration clusters (524 
of the 1,685) and 21 percent of the low mmority 
clusters (701 of the 3,315). This difference between 
low and high minority clusters may be associated with 
the tendency of high minority clusters to be located in 
urban areas, which tend to have a higher within-cluster 
household density. 

Since the retained sample of residential clusters 
exceeded the targeted number for both types, a sample 
of 337 of the 524 retained high minority clusters and 
663 of the 701 low minority clusters were randomly 



selected and used as the first stage sample. This 
oversampling of clusters was accounted for in the 
development of sao^ling weights. 

Sampling Telephone Numbers and Identifying 
Eligible Households 

After the 1,000 residential clusters were 
selected, the next step was the identification of 
residential households for a sample of the telephone 
numbers within the clusters. A sample of 30 of the 
telephone numbers within each of the 1,000 clusters 
was randomly selected. Additional telephone numbers 
were selected from within the cluster if the number of 
households identified in the cluster was small (fewer 
than four). The purpose of adding these numbers was 
to reduce the variability in the weights. A total of 590 
additional telephone numbers were included in the 
NHES Field Test as a result of this procedure. The 
total number of telephone numbers dialed was 30,590. 

The expected within-ciuster residency rate (the 
number of residential households in a cluster divided by 
the 100 numbers in the cluster) for a RDD survey using 
this design is between 60 and 65 percent. Of the 
30,590 numbers that were dialed for the NHES Field 
Test, 60 percent v/ere identifier as residential, 36 
percent were identified as nonresidential, and 4 percent 
could not be classified by residential status (table 1). 
The unclassified numbers are telephones that were 
dialed at least 7 times (almost all were dialed 14 times) 
over a varied schedule of daytime, nighttime, and 
weekends but were never answered. If these numbers 
are proportionally allocated by the residency rates 
associated with the numbers that were classified, then 
the estimated within-cluster residency rate is 63 percent. 
The number of residential telephone numbers that 
participated (i.e. , someone in the household was willing 
to complete the screening items needed to determine if 
there were household members who might be eligible 
for NHES) is also shown in the table. 

Three different methods for computing 
response rates for this screening stage were used (table 
1). The three methods differ in how they handle the 
telephone numbers that were never classified by 
residency status. The numerator for all three screener 
response rates is the number of telephone numbers that 
were identified as residential and participating. The 
denominator for the conservative estimate is the sum of 
the residential, participating and not participating 
numbers and the unknown residency status numbers. It 
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Table 1. -Number of telephone n^nbers dia.ed, residential status, and screener response rates 



Ci^rAAnar fAorit^nCO f^HtOnrtrw 

ocreener rcopurioc uaicyuiy 


Number 


Percent of 
ail numbers 


Percent of 
residential 


Total 


30,590 


100.0 




Identified as residential 


18,406 


6>'>.2 


100.0 


Participating 
Not participating 


15,037 
3,369 


49,2 
11.0 


81.7 
18.3 


Identified as not residential 


11,041 


36.1 




Unknown residential status 


1,143 


3.7 




Screener response rates^ 


Rate 


Estimated (CASRO) response rate 




78.6% 




Conservative response rate 




76.9 




Liberal response rate 




81.7 





^Ail three response rates were computed using the number of participating households as the numerator. The 
denominator for the conservative rate Includes the numbers with unknown residential status. The denominator 
for the liberal rate does not include the numbers with unknown status. The denominator for the estimated 
(CASRO) rate includes a proportion (based upon the residency rate for the known numbers) of the nun bers with 
unknown residential status. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) Field Test. 
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is conservative because it assumes that all the unknown 
residency status numbers are residential. For the Field 
Test, the conservative screener response rate is 77 
percent. The liberal estimate takes the opposite 
approach. The denominator for the liberal rate 
excludes the unknown residency status numbers, 
assuming they are not residential numbers. The 
resulting response rate for the liberal approach is 82 
percent. 

The actual response rate is most likely between 
these two extremes. The estimated response rate is a 
compromise estimate that assumes that the numbers 
with imknown residency status have the same 
distribution as the numbers for which residency status 
is known* The effect of this assumption is to estimate 
that about 63 percent of the unknown numbers are 
residential and include them in the denominator. The 
screener response rate estimated in this manner is 79 
percent. The estimated screener response rate is 
consistent with the method suggested by the Coimcil of 
American Survey Research Organizations (CASRO).^ 

The CASRO response rate probably 
underestimates the true response rate. Several studies 
that have examined telephone numbers with unresolved 
residency status (including checks with telephone 
companies) have found that the majority of these 
numbers are nonresidential. In another RDD study, the 
1987 Adult Smoking Survey, only about one-third of 
the unresolved numbers were found to be residential. 
This fmding is consistent with studies such as in the 
National Crime Survey test^° in which 38 percent of 
unresolved numbers were residential. 

Table 2 provides greater detail on the 
participating and nonparticipating residential telephone 
numbers. The participating households are divided into 
out-of-scope and in-scope households. A household 
was in-scope if it contained a person between 2 and 6 
years of age, a person between 13 and 22 years of age, 
or a female aged 28 to 65 years in a household selected 
for the multiplicity sample. These broader age ranges 
were utilized because Screener respondents are 
sometimes in error about the current age of household 
members. The final eligibility requirements for the 
survey, based on the actual month and year of birth 
obtained from the appropriate respondent, were more 
restrictive than these initial Screener requirements; 
therefore, more than 1,000 of the initially eligible 
households were ultimately found to be ineligible for 
the survey. 
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Unweighted counts of telephone numbers v.'ere 
used to compute these response rates instead of counts 
weighted by the inverse of the probability of selection. 
There are two justifications for this method. First, the 
full probability of selection depends on factors (such as 
the number of telephones in the household) that are not 
known for nonresponding numbers. Second, all the 
telephone numbers were selected witli equal probability 
(in their high and low minority strata) so that weighted 
and unweighted rates are nearly identical. This finding 
is discussed below. 

The vast majority (84 percent) of the 
residential households which did noi participate in the 
survey were refusals. Because of the sparsity of data 
on households that do not participate in RDD surveys, 
it is difficult to analyze the response patterns associated 
with refusals. One important variable that is available 
for all households is the classification of the telephone 
number as being in a high or low minority- 
concentration cluster. For the Screener, the percent of 
telephone numbers that were refusals was nearly 
identical for the high and low minority clusters (16.3 
percent and 15.7 percent, respectively). Although this 
sample indicator is very limited, it does suggest that 
minority residence within a cluster is not a source of 
differential response bias. We would suspect that if 
response rates differed drastically by race, then the 
response rates of high and low minority clusters might 
be somewhat different. 

A Spanish langxiage version of the instrument 
was used in order to reduce the number of language 
problem cases. Cases identified as language problems 
were followed up by interviewers fluent in Spanish. 
Interviews were completed in Spanish for 244 
Screeners, 87 Household Respondent Interviews, 83 
Youth Interviews, and 59 Parent Interviews. (All of 
these cases were coded as initial language problems, but 
the followup interviews may have been done in English 
for some of the cases.) If members of the household 
did not speak English or Spanish, or the persons could 
not be reached for a Spanish language interview, the 
case was coded as a final language problem. 

The maximum call category contains the 
telephone numbers of households that failed to respond 
to the screener despite at least seven attempts over a 
specified variety of times. These numbers were 
classified as residential numbers, since their status was 
known from the contacts made. For example, a contact 
with a household may have been made, but an adult 
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Table 2. — Number and percentage of telephone households, by Screener response category 



Screener response category 


Number 


Percent 


Participating residential phone numbers 


15,037 


100.0 


Out-of-scope households 


9,589 


63.8 


}n-scope^ households 


5,448 


36,2 


Not participating residential phone numbers 


3,369 


100.0 


Refusals 


2,832 


84,1 


Language problems 


189 


5.6 


Other problems 


112 


3,3 


Maximum calls 


236 


7,0 



^In-scope households are those which satisfied the more strict screening age eligibility requirements. Only 4,374 
of the 5,448 households which met the screening requirements actually contained a person eligible for the survey. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) Field Test. 
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household member eligible to answer the Screener was 
not available and could not be reached on subsequent 
attempts. The "other problems" category was used to 
capture miscellaneous categories of nonresponse, such 
as respondents who moved after initial contact, 
respondents who would not be available to participate 
until after the survey period ended, and cases which 
were only one or two calls short of being coded in the 
maximum call category when data collection was 
terminated. 



Classification of Eligible Households 

A household could be eligible for the Field 
Test if either a 3- to 5-year-old resided in the household 
or a 14- to 21 -year-old civilian was a member of the 
household. We obtained a sample of 3,190 households 
with 14- to 21-year olds, including 2,987 with this 
population only and 203 also containing 3- to 5-year- 
olds (table 3). The number of households with 14- to 
21-year-old youth was close to the expected number of 
3,330, based on estimates from the October 1988 CPS. 
We obtained a sample of 1,387 households with 3- to 5- 
year-olds, of which 203 also contained 14- to 21-year 
olds; the expected number was 1,050. The percentage 
of households with members in both age ranges was 5 
percent, slightly greater than the expected 2 percent. 
Table 4 shows the distribution of the number of eligible 
households, by type of household members eligible for 
the survey and the number of eligible persons per 
household. The number of 3- to 5-year-olds in 
households with both types of eligible persons is shown 
in the parentheses in the last column. The figures 
shown include the multiplicity sample. In this 
approach, the out-of-household 14- to 21 -year-old 
sampled women were eligible for sampling. This 
approach, which was utilized in one of four households, 
increased the number of 14- to 21-year-olds by 199. 

Among participating households, 21.2 percent 
had at least one 14- to 21 -year-old— slightly lower than 
the 22.6 percent expected, based upon the 1988 CPS 
data. About 9.2 percent of participating househclus had 
3- to 5-year-olds, again slightly below the 10.4 percent 
expected, based on the 1988 CPS data. The average 
number of eligible persons per household was 1.1 for 
3- to 5-year-olds and 1.4 for 14- to 21 -year-olds. 
These averages matched the CPS estimates for the 
number of members per household in households with 
such members. 



Obtaining Interviews for Eligible Persons 

Although several different interview processes 
were used in the Field Test, three potential interviews 
are of particular interest to analysts of the data. One is 
tlie Parent Interview with the parent or guardian of each 
3- to 5-year-old in the household. The second is the 
Household Respondent Interview (HRI) with a person 
(possibly the youth him- or herself) knowledgeable 
about the educational status of each 14- to 21 -year-old 
in the household. The third is the Youth Interview, 
conducted with the youth him- or herself for a sample 
of the 14- to 21-year-old household members. 

Youth Interviews were conducted for each 14- 
to 21-year-old identified by the household respondent as 
a dropout. These youths are called certainty selections. 
For other 14- to 21-year-olds, a sample of about one in 
five youths (called noncertainty selections) were chosen 
for administration of the youth interview. The total 
sample size for each category of youth interview is 
shown in Table 5. The number of certainty selections 
(1,158) was much larger than the anticipated sample of 
30O persons. This occurred because the sampling 
algorithm erroneously included as certainty cases 
persons with a high school diploma who were not 
currently enrolled in an educational program. These 
youth were included as certainty cases along with 
dropouts. As a result, the classification of persons as 
certainty and noncertainty cases does not have much 
substantive importance. 

Because of the ambiguity in the classification 
of case*, by certainty status, a direct comparison to CPS 
estimates of dropouts was made to evaluate the yield of 
the NHES. Based upon the 1988 CPS, the number of 
status dropouts expected from a sample of 3,190 
telephone households with 14- to 21 -year-olds is about 
322. The number obtained in the NHES was 319. 
While the estimate of the expected number of cases 
does not take into account the oversampling of high 
minority clusters, these figures do indicate that the 
sample yield is about what would be expected. The 
oversampling would increase the expected values 
slightly. 

Table 5 also shows the number of sampled 
persons for each category of household member and the 
number of persons who completed the interview. Tiie 
data include the cases in the multiplicity sample as well 
as those in the sample households. The percentage of 
sampled persons who responded to each interview is 
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Table 3.-Number and percentage of participatina eligible households, by type of eligible person 



Type of eligible person 


Number of 
households 


Percent 


Total 

Only 3- to B-year-olds 

Only 14- to 21 -year-olds 

Both 3- to 5-year-olds and 14- to 21 -year-olds 


4,374 
1,184 
2,987 
203 


100,0 
27.1 
68.3 
4.6 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National 
Education Survey (NHES) Field Test. 
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Table 4. -Number and type of participating, eligible households, by number of eligible persons in household 



Number of eligible 
persons per household 


Number of participating households with eligible persons 


Only 3- to 
5-year-olds 


Onlv 1 4- tn 

21 -year-olds 


3- to 5-year-olds 
and 14- to 
21 -year-olds^ 


Total 




1,184 


2,987 


203 


(226) 


1 
2 




1,046 


2,030 


0 


(0) 


3 




135 


779 


125 


(125) 


4 




3 


143 


62 


(78) 


5 




0 


31 


14 


(19) 


6 




0 


3 


0 


(0) 


7 




0 


1 


1 


(2) 






0 


0 


1 


(2) 



•The number of 3- to 5-year-olds in these households is given in parentheses after the count of households. 

SOURCE: as. Department of Education, National Center for Education Statistics, 1983 National Household 
Education Survey (NHES) Field Test. '-duund, nousenoia 
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Table 5. -Number of sampled and completed interviews, completion rates, and response rates, by type of 
interview 



Type of Interview 


Number 


Completion 
rate 


Overall 
estimated 
response rate 


Parent Intorv/ioiA/c ahmit tn R-v/PJif-nlHc 








oampieo lor inicrvicw 
Completed interviews 


1 ;551 

1,530 


98.6% 


77.5% 


UJrki leoKrklH RocnnnHont IntPfuiPiA/Q ^ihniit 1/1- tn 9 1 -\/P?ir- 

nUUoCllUlU fiCopUl lUCI 1 1 HIlCIVICWo OUUUl 1 *T \.\J \ yCOi 

oids 








Sampled for interview 
Completed interviews 


4,441 
4,313 


97.1 


76.3 


Youth Interviews 








All interviews 








sarTipicu lor iiiicivjcw 
Completed interviews 


1 ,863 
1,604 


86.1 


65.7 


Certainty selections 








Sampled for interview 
Completed interviews 


1,158 
985 


85.1 


64,9 


Certainty selections 








Sampled for interview 
Completed interviews 


705 
619 


87.8 


67.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) Field Test. 
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called the completion rate-— a conditional response rate 
indicating the percent of interviews that were completed 
if the household was screened successfully. The 
completion rate was 99 percent for the Parent/Guardian 
Interview, 97 percent for the HRI, and 86 percent for 
the Youth Interview. 

The overall response rate for the 
Parent/Guardian Interview and the HRI was the product 
of the Screener response rate (79 percent) and the 
appropriate completion rate. These overall response 
rates were 78 and 76 percent, respectively. The overall 
response rate for the 14-to 21 -year-old Youth Interview 
had an additional component, since this interview could 
not be conducted unless an HRI was completed. The 
overall response rate of 66 percent for the Youth 
Interview was, therefore, the product of the Screener 
response rate, the HRI completion rate, and the Youth 
Interview completion rate. 

A potential source of bias in a survey is 
differential response rates for key population domains. 
Completion rates for the HRI and the Youth Interview 
were also calculated by the domains used in estimation 
(table 6). The domains indicated the presence of a 
telephone in the household of the youth and the youth's 
mother and whether or not the youth lived in the same 
household as the mother. An additional domain, which 
overlapped the others, indicated whether the youth was 
sampled tlirough multiplicity sampling. Considerable 
variation by domain could introduce bias into the results 
if it were not taken into account in the estimation 
process. The completion rates for the HRI were 
consistently high, in the range of 93 to 98 percent, for 
all domains. 

For the Youth Interviews, the completion rates 
were lower for youths who did not live in telephone 
households (41 percent) and those who were sampled 
by multiplicity sampling based on the reports of their 
mothers who live in telephone households (50 percent). 
It was very difficult to locate and conduct telephone 
interviews for out-of-household youths sampled in this 
way, especially those who do not have a telephone in 
their household. 

Although the completion rates for the Youth 
Interview were low, they are higher than was expected 
prior to the Field Test. Phone numbers of friends, 
work phone numbers, and phone numbers of other 
relatives were used to help locate and interview youths 
without home telephones. Including even 40 percent of 
the persons who othenvise would be outside the scope 



of a telephone survey may have significant implications 
for bias reduction. Furthermore, HRI's were 
completed for 97 percent of the multiplicity sample 
youdis. The evaluation of the multiplicity sampling 
includes these completion rates and ' er factors, such 
as the usefulness of the household respondent data. 
(For more information on multiplicity sampling, see 
Multiplicity Sampling for Dropouts in the NHES Field 
Test.) 

Comparison of NHES Response Rates with Other 
Surveys 

In order to help evaluate the unit response rates 
of the NHES Field Test, these rates were compared 
with those obtained in similar RDD telephone surveys 
and with NCES statistical standards for response rates. 
NCES Standards^' indicate that the target response rate 
for a cross-sectional survey should be at least 85 
percent. For hierarchically selected surveys, the target 
response rate applies to each hierarchy. For example, 
a survey of students within schools would have target 
response rates of 85 percent for both the schools and 
the students, resulting in a final response rate of 72 
percent. 

Applying this standard to NHES, the response 
rate for the Screener was below the target for a cross- 
sectional survey. If, however, the NHES is considered 
a hierarchical survey (sampling households and then 
persons within households), the response rates for both 
the HRI and the Parent/Guardian Interviews exceed the 
required standard of 72 percent. 

Other U.S. Government RDD surveys have 
generally reported response rates in the same range as 
the Field Test. For example, Durant and Vitrano'^ 
reported response rates between 70 and 74 percent for 
the RDD portion of a survey conducted by the Census 
Bureau for the Bureau of Labor Statistics' Point of 
Purchase Survey. Alexander, Sebold, and Pfaff^ 
reported response rates of between 75 and 80 percent in 
a test of RDD by the Census Bureau for the National 
Crime Survey. For the National Center for Health 
Statistics' Telephone Household Interview System 
(THIS), Fitti'^ reported response rates between 72 and 
85 percent in the first years of the system's 
implementation. About 8 years later, tlie response rates 
were reported to be about 80 percent. 

The completion rates for the extended 
interviews, conditioned on the completion of the 
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Table 6, -Number of youth sampled, interviews completed, and interview completion rates, by type of 
interview and domain^ 



lypc Ul IMlCiVlCW ai lU uuiiiaiii 


• 

Number 


Number 


Completion 


sampled 


completed 


rate 


HnncPhnlH RpcnnnHpnt IntPrviPWS 








Domains 








Youth in same telephone household as mother 


3,321 


3,246 


97,7% 


Vr\i i+h in tolonhnno hniicPhnlH mothPr 

TUUlli ill lClC|JllUllC IIUUoCIIUlU/ IIIWIIICI 




162 


93,1 


not in telephone household 


174 






Youth in telephone household, mother 




o4o 


yo,b 


in rliffprpnt tplpnhnnp household 

III VJlll^l^llL i.^l^Ly|IWIiV-* iixy\^ijv^iivyi\j 


887 




96.6 


Vniith nnt in tplpnhnnp hnu<;Phold 

Tv/ULII lIUL III LCiC^IIUIIC i iwuoc^i iwiVJ, 




57 


mnthpr in tplpnhnnp hnu<;Phold 

lliULIlCI III LCIC^IIUIIC IIWUO^IIWIVJ 


59 






Multiplicity sample youth^ 


199 


192 


96.5 


Ymith IntPrviPw** 

lUiJlil IIILCylVICyWO 








Domain*; 

^^\J 1 1 1 0 1 1 lO 








Youth in same telephone household as mother 


1,224 


1,076 


87.9 


Youth in telephone household, mother 


101 


89 


88.1 


not in telephone household 






84.2 


Youth in telephone household, mother 


500 


421 


in different telephone household 






40.9 


Youth not in telephone household, 


44 


18 


mother in telephone household 








Multiplicity sample youth^ 


133 


67 


50.4 



^The term domain refers to the population segment to which the youth belongs. These domains represent 
segments with different coverage rates and different probabilities of selection. 

^he counts of the multiplicity sample youth overlap with the counts in the other domains. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) Field Test. 
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Screener, were excellent. These findings suggest that 
efforts should be concentrated on increasing the 
Screener response rates for future surveys. 

The extent of the problem associated with 
refusals, especially for the Screener, is evident (table 
7). About 84 percent of all nonresponse for the 
Screener was the result of refusals, with the remainder 
including maximum contact without completion, 
language problems, and other problems. The refusals 
were largely comprised of breakoffs (i.e., interviews 
that were begun but not completed because the 
respondent refused to continue at some point during the 
interview). These breakoffs account for 45 percent of 
all Screener nonresponse. 

Breakoffs in interviews sometimes suggest 
problems associated with particular items in the 
instrument. To examine this possibility the number of 
breakoffs for each item on the instrument was tabulated 
(table 8). The Screener items are listed in the order in 
which they appear in the interview because breakoffs 
frequently occur in the first few items in an instrument. 

The counts of breakoffs are clustered in two 
areas, About one-third occurred in the first two items 
of the Screener, The number of breakoffs in the initial 
items of the Screener was not large relative to 
expectations (for example, in the other three interviews 
the percent of breakoffs in the first three items 
exceeded 80 percent), which suggests that there were no 
particular problems with the items. Another third of 
breakoffs were encountered in the items used to 
enumerate the members of the household. (NHES 
interviewers indicated that these items caused them the 
greatest problem during the interview). These results 
suggest that alternatives to full household enumeration, 
or to enumeration at the start of the survey, should be 
considered as a means of improving the response rates 
for the Screener in future NHES collections. 



Item Response Analysis 

Persons who respond to a survey may not 
complete all of the items in all of the questionnaires 
they are presented. This type of nonresponse— ittm 
nonresponse — is a problem because it introduces bias 
into estimates just as unit nonresponse does. In the 
extreme, high item nonresponse rates can prevent the 
estimation of important characteristics from a 
survey.'* For example, if a substantial proportion of 
respondents have missing values for a key variable. 
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analyses including that measure would be incomplete or 
could be eliminated entirely. 

Item nonresponse may occur for several 
reasons. One reason is that particular items, such as 
questions about household income, may be considered 
by some respondents to be too personal to answer. 
Another reason, most frequently encountered in lengthy 
interviews, is that the respondent breaks off the 
interview and does not finish it later. Item nonresponse 
may also occur in a complex survey as the result of 
faulty skip patterns in the interviews. If a skip pattern 
is incorrect, some people may never be given the 
chance to respond to a particular item. Additional 
reasons for item nonresponse are the failure of the 
interviewer to record the respondent's answer and the 
inability of the respondent to answer the question. 

The use of CATI helped to eliminate some of 
these problems. One of the innovative features of the 
CATI system used for the NHES is that it enabled 
interviewers to enter and modify a matrix of data 
without having to return to previous screens to ask 
questions again for additional household members. 
This feature was especially useful in easing the response 
burden during household enumeration. 

Finally, the CATI system also helps to prevent 
item nonresponse because it will not permit an 
interviewer to skip an appropriate item without entering 
a response. Nor will it permit the entry of an out-of- 
range response without confirmation of the value 
entered. 

Another aspect of the CATI system that helped 
to reduce item nonresponse was the system of pre- 
determined skip patterns that prevented interviewer 
error resulting in failure to ask appropriate questions. 
Extensive testing of the system's skip patterns by 
several types of staff members helped lo ensure that 
they were working properly prior to the beginning of 
data collection, as did the pretest of the CATI system 
with a small household sample. 

An investigation was made of the approximate 
size of the item completion rates in the Field Test. The 
item completion rates are examined for selected items 
from each of the four main instruments— the Screener, 
Household Respondent Interview, Youth Interview, and 
Parent Interview. 

An item is considered to have a valid response 
if the respondent provided an answer that was in the 
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Table 7. —Number and percentage of interviews not completed, by instrument 



Instrument 


Interviews not completed 


Total 
number 


Refusals 


Types of refusals 


Breakoffs^ 


Other^ 


Screener 


3,369 


84.1% 


45.3% 


38.7% 


Household Respondent Interview 


128 


55.8 


17.8 


38.0 


Youth Interview 


259 


35.4 


9.3 


26.0 


Parent Interview 


21 


65.6 


29.7 


35.9 



interviews that were begun but not completed because respondents refused to continue at some point during 
the interview. 



^Includes language problems, maximum calls, and miscellaneous categories of nonresponse. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) Field Test. 



Table 8. -Number and percentage of breakoffs in Screener, by itenn 



Screener item 


Breakoff^ 


listed in order of appearance in interview 


Number 


Percent 


Total 


1,527 


100.0 


Phone for home or business use 


413 


27.0 


Other phone in home 


118 


7.7 


Other phones for home or business use 


8 


.5 


Other household served by phones 


107 


7.0 


Respondent household member over 1 7 


68 


4.5 


Any household member over 17 


9 


.6 


Ask for household member over 1 7 


12 


.8 


Speak to head of household 


2 


.1 


Any household members under 65 


108 


7.1 


introduction to matrix 


132 


8.6 


Enumeration matrix 


525 


34.4 


County 


6 


.4 


Kind of home 


2 


.1 


income 


11 


.7 


Any out-of-household youth of women 


2 


.1 


Know about education of youth subject 


1 


.1 


Name of parent/guardian 


3 


.2 



^Breakoffs are interviews that were begun but not completed because respondents refused to continue at 
point during the interview. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) 
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acceptable range specified for that item. If the 
respondent refused to answer, did not know the answer, 
or the answer was not ascertained, the result is 
considered a nonresponse. The response "did not know 
the answer" may be a valid response. The inclusion of 
this category in the missing values tends to overstate the 
problem of missing responses. If the item was skipped 
in an interview as a part of a legitimate skip pattern, the 
item for that interview is not included in either the 
numerator or denominator of the item completion rate. 

Because of the large number of items in each 
of the interviews, only some were selected for 
presentation. Included were the following: 1) items of 
high substantive importance for the particular interview; 

2) items with relatively high rates of nonresponse; and 

3) items that appear at both the beginning and end of 
the interview. 

The item completion rates described in the 
followmg subsections are generally very high. Most 
items, including those at the end of the interview, have 
completion rates close to 100 percent, indicating that 
respondents did not "break ofP the interviews due to 
their length. Because of the high item completion rates, 
the tabulations of item completion rates by demographic 
and other characteristics of the respondents have not 
been examined. Item completion rates are indicative of 
the willingness and ability of persons to respond to the 
particular questions asked. They do not reflect the 
validity or accuracy of those responses, and the overall 
s-uccess of the data collection cannot be measured 
without measures of external validity. 

Screener Item Completion Rates 

In the Field Test, the Screener was used to 
determine eligibility for the survey by enumerating the 
members of the households and to collect data on 
selected household characteristics. The determination 
of eligibility of the households was essential for 
completing other interviews in NHES. Any household 
for which the information needed to establish eligibility 
was not obtained was considered a unit nonrespondent. 
Therefore, the set of items in the Screener for which 
item nonresponse did not preclude inclusion in the 
survey was limited. 

Table 9 shows the number of households in 
which the selected items on the Screener were 
attempted, those for which a legitimate response was 
obtained, and the resulting percent of attempts that were 



completed. The item completion rates for the first four 
items in the table are nearly 100 percent. Of these, the 
items addressing additional phone numbers and home 
tenure are important because they are used in the 
weighting and estimation process and nearly complete 
response is essential. 

In the Field Test, a test of approaches for 
asking the income item was conducted. Approximately 
half of the sample was asked to report the actual 
household income, while the other half was asked to 
indicate which of several income ranges best described 
their household income. Those households that refused 
to respond to the specific income item were asked the 
income range item as a followup. 

The income item falls in the valid range for 88 
percent of all households. Of the sample of households 
that were first asked the specific income version, 65 
percent gave a valid response. By administering the 
followup item, 65 percent of those who did not 
complete the question were converted to completions. 
As a result of this two-step procedure, a valid response 
was obtained for 86 percent of this sample of 
households (not shown in table). Of the sample of 
households that were first asked the income range 
version, 89 percent completed the item. 

The item nonresponse to the specific income 
question produced a bias in the resulting data. Those 
who did not respond to the item appear to be 
disproportionately from lower income households. The 
resulting distribution, when compared to those asked the 
income range question, shows lower proportions of low 
income household, and higher proportions of moderate 
and high income households (figure 2). 

The distribution of incomes for those who did 
not respond to the specific income item, but did 
respond to the ranges when they were used as a 
followup question, is shown in figure 3. The 
proportion of this group that report income in the two 
lower categories is substantial, although all income 
levels are represented among this group. 

These results suggest that unless the specific 
income item is required for analytic uses, the income 
range item may be a preferable way to collect income 
data. Examination of the income values reported under 
the two procedures revealed a bias when the specific 
income item was used. Low income households had a 
much lower item response rate than did high income 
households. Therefore, even if specific income is 
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Table 9.— Item response rates for selected items from the NHES Screener 



Screener item 


Number 
attempted 


Number 
completed 


Percent 
completed 


Additional phone numbers 


5,441 


5,438 


99.9 


Additional phone users 


5,441 


5,435 


99.9 


Home type 


5,441 


5,436 


99.9 


Home tenure 


5,441 


5,420 


99.6 


Income of household (all) 


5,441 


4,774^ 


87.7 


Income amount item 


2,657 


1,730 


65.1 


Income range item 


2,784 


2,476 


88.9 



^Includes 568 of the 927 respondents who refused when asked the actual amount item but provided income 
when it was asked as a range. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey Field Test. 
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Figure 2. - Distribution of responses to specific and range income items 




under SIO.OOO $10,001-20,000 $20,001-40,000 over $40,000 



Reported household income 



Figure 3. - Distribution of responses for those who did not answer the specific 
income item, but responded to the range item 




under 510,000 $10,001-20,000 $20,00140,000 over $40,000 



Reported household income 
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needed in the analysis, there should always be an 
income range followup item to reduce the bias 
associated with the nonresponse to this item. 

Household Respondent Interview (HRI) Item 
Completion Rates 

The HRI was used to obtain data about the 
educational status of each 14- to 21 -year-old in the 
household. The data from this interview were also used 
in sampling persons for the Youth Interview. The HRI 
was conducted with an adult in the household who was 
knowledgeable about the educational status of the 14- to 
21 -year-old residing in the household. If the respondent 
was unable to give both the birth date and the current 
enrollment status of the youth, that person was not 
eligible to serve as the household respondent for the 
youth. The 14- to 21-year-old could be the household 
respondent for him- or herself, provided the youth was 
at least 18 years old or there was no other adult 
member of the household. 

The rules used in defming a respondent for the 
NHES can be compared with the ones used in the CPS. 
In the CPS, the interviewer is encouraged to interview 
the most knowledgeable aduit (at least 14 years old) 
household member, although any adult member of the 
household is eligible and proxy respondents can be used 
in some circumstances. The rules used to define the 
respondents and the impact of proxy respondents in the 
CPS are described in Brooks and Bailar.'^ 

HRIs with the mothers of out-of-household 14- 
to 21-year-old youths identifier! in the multiplicity 
sample are included in the item ajmpletion rates. This 
inclusion is appropriate because the mother was asked 
to complete an entire interview for these identified 
youths. The completion rates for these interviews are 
examined in greater detail in the evaluation of the 
effectiveness of multiplicity sampling in Multiplicity 
Sampling for Dropouts in the NHES Field Test. 

Table 10 gives the item completion rates for 
selected items from the Household Respondent 
Interview. The number of items attempted varies 
across the selected items because of legitimate skip 
patterns. For example, the current educational program 
item was only asked for those persons currently 
enrolled in some program. The item on current 
services for the handicapped was asked only for those 
persons who have a handicap that affects their ability to 
perform in school. 
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The current enrollment status has a 100 percent 
item completion rate. As mentioned previously, this 
item was required in order to initiate this interview. If 
no interviewed person in the household was able to 
provide a response to this item, the HRI for the youth 
was considered a imit nonresponse. The item 
completion rates for almost all items exceed 99 percent. 
One exception is the item concerning the youth's main 
activity during the last week, which classified a youth 
as being in or out of the labor force. It is only asked 
for those youths who are not currently in an educational 
program and are not currently working for pay. About 
87 percent of the qualified respondents provided a 
legitimate response for this item. 

Youth Interview Item Completion Rates 

The Youth Interview was conducted with the 
14- to 21 -year-old him/herself for a sample of the 
youths identified in the HRI. If a youth also completed 
the HRI, those items on the Youth Interview that were 
identical to items on the HRI simply were transferred 
into the appropriate locations on the Youth Interview 
record. Because of the relatively small sample size and 
skip patterns, the number of attempts is limited for 
some of the items in the Youth Interview. The 
completion rates for most items are excellent; in almost 
all cases the rates exceed 98 percent (table 11). 

The item about main activity last week had the 
lowest item completion rate from the selected items 
presented on the HRI; at 88 percent, it is almost equal 
to the 87 percent response to this item in the HRI. Of 
the items selected for analysis, the one asking if the 
youth expects to continue his or her schooling has the 
lowest item completion rate— 85 percent. A plausible 
explanation for this relatively low item completion rate 
is that the youth may not have decided on future plans 
regarding education. In this case it could be argued 
that "undecided" or "don't know" is a legitimate 
response, rather than a nonresponse. In fact, only one 
of the 142 youths refused to answer this item. About 
half of the nonrespondents said they did not know their 
plans, and an answer was not ascertained for the other 
half. Thus, if undecided or don't know is considered 
a valid response, the item completion rate for this item 
would be 92 percent. 



Table lO.-ltem response rates for selected items from the fJHES Household Respondent Interview 



Household Respondent Interview item 


Number 
attempted 


Number 
completed 


Percent 

completed 


Enrollnnent status - current 


4,313 


4,313 


100.0 


Educational progrann - current 


3,225 


3,223 


99.9 


Regularity of attendance - current 


2,036 


2,034 


99.9 


Grade or year - current 


2,036 


2,034 


99.9 


Enrollment status - last year 


4,313 


4,300 


99.7 


Educational program - last year 


3,538 


3,538 


100.0 


Regularity of attendance - last year 


2,618 


2,598 


99.2 


Grade or year - last year 


2,618 


2,61 1 


99 7 


Diploma status 


2,277 


2,611 


99.6 


Diploma type 


1,961 


1,961 


100.0 


Handicapping condition - current 


4,313 


4,301 


99.7 


Handicapped services - current 


117 


117 


100.0 


Employed - current 


4,313 


4,280 


99.2 


Main activity last week 


119 


104 


87.4 


Education of mother 


4,313 


3,974 


97.1 



SOURCE: U.S. Department of Education, National Center 
Education Survey Field Test. 



for Education Statistics, 1 989 National Household 
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1 1 Hrrr r-n— -tP. for ..elected items from the NHES Youth Interview 








Number 


Number 


Percent 




Youth Interview item 


attempted 


completed 


completed 




Enrollment status - current 


1,604 


1,604 


100.0 1 




Educational program - current 


683 


883 


100.0 




Regularity of attendance - current 


437 


433 


99.1 




Grade or year - current 


437 


437 


100.0 




Enrollment status - last year 


1,604 


1,603 


99.9 




Educational program - last year 


1,052 


1,052 


100.0 




Regularity of attendance - last year 


706 


702 


99.4 




Grade or year - last year 


706 


702 


99.4 




Expect to continue schooling 


921 


779 


84.6 




Diploma status and type 


1,167 


1,167 


100.0 




Reason for leaving regular school 


305 


291 


95.4 




Handicapping condition - current 


1,604 


1,591 


99.2 






13 


12 


92.3 




Services - current 






Stopped attending - ever 


1,604 


1,591 


99.2 




Employed - current 


1,604 


1,587 


98.9 




Main activity last week 


102 


on 


88.2 




People in household now 


1,604 


1,587 


98.9 




Number of siblings 


1,604 


1,585 


98.8 




Education of mother 


1,604 


1,472 


91.2 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National 
Education Survey Field Test. 
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Parent/Guardian Interview Item Completion Rates 

The Parent/Guardian Interview was used to 
collect information about the care and education of 3- 
to 5-year-olds. In order to be a respondent for the 3- 
to 5-year-old, the person must be able to provide the 
birth date aiid say that he or she was the person who 
knew most about the care and education of the child. 
For 95 percent of the children for whom interviews 
were completed, either the mother or the father was the 
respondent; about 80 percent of the respondents were 
the mothers. 

As with the other instruments, the number of 
respondents for whom an item was attempted varies 
depending upon skip patterns. All of the completion 
rates for the selected items exceed 98 percent (table 
12). 

The item on the number of days per week that 
the child attends care/preschool is asked for each child 
care/preschool arrangement involving the child. The 
number of times the item was attempted includes each 
time the item was a.sked for each anangement, and 
therefore includes multiple attempts for children with 
multiple care/preschool arrangements. The high item 
completion rates for the Parent/Guardian Interview are 
very important, especially because this interview took 
the longest time to complete and was the most complex. 
The item completion rates, when evaluated in 
conjunction with the unit response rates and the time 
required to complete the interviews, indicate the success 
of the NHES Field Test interview process. 



Survey Timing 

The length of time it takes to complete an 
interview is an important factor in all surveys, 
especially those conducted via telephone. While the 
survey should include all the important analytic 
variables, it should also be as short as possible to 
reduce the burden on the public and encourage complete 
and reliable response. Interview length can also be 
used to measure productivity and to plan future studies 
using similar items. A discussion of the effects of the 
length of the interview on response quality is given in 
Herzog and Bachman.'^ 

For the NHES Field Test, the amount of time 
it took to complete various segments of each of the four 
major interviews was recorded in the CATI itself. If 
unedited, these timings could be misleading as a 



measure of burden on the public for several reasons. 
One major reason is that some interviews have 
discontinuous segments (i.e., a segment that is begun 
but not completed until a later date or time because of 
respondent availability). The CATI system was 
programmed to eliminate the timing variables for 
discontinuous interview sessions for this reason. 
Accordingly, the timings for about 2 percent of the 
HRIs, 8 percent of the Youth Intei-views, and 4 percent 
of the Parent/Guardian Interviews were eliminated 
because of discontinuous interviews. 

The unedited length of time recorded for the 
segment might be misleading for other reai:ons. For 
example, if the interviewer was forced to wai^ on the 
telephone while the respondent took care of other 
business (e.g., to attend to a young child or Uike 
another call), then the recorded length of time for thj*. 
segment would reflect more time than was actually 
required for the interview. While this additional time 
distorts estimates of the burden on the respondent, its 
inclusion is appropriate if the timings are being used to 
estimate interviewer hours. 

Since the purpose of this section of the report 
is to estimate respondent burden for the NHES 
interviews, the length of time for each segment was 
truncated to exclude extremely unreasonable values 
prior to the analysis. Consequently, the timings 
reported in this section should not be used for 
estimating interviewer time. Only those interviews for 
which all component times were within specified 
reasonable ranges (both upper and lower ranges) were 
retained. Flow diagrams for each instrument include 
the valid ranges for each segment of each interview 
(figures 4 to 6). If a segment for an interview was out 
of range, the timings for all segments of the interview 
were excluded. About 11 percent of the HRIs, 14 
percent of the Youth Interviews, and 13 percent of the 
Parent/Guardian Interviews were excluded because of 
invalid times, but these exclusions did not have a 
substantial impact on the results of the length 
analysis.'' 

Table 13 shows the mean total times to 
complete the instruments before and after exclusion of 
items with invalid times. The mean times for all cases 
are only slightly greater than the mean times for the 
cases excluding those with invalid times. Since these 
values are so close, and the mean is relatively sensitive 
to outliers (one very large value can have a big 
influence on the mean), it is very unlikely that the 
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Table 12. -Item respoHsSe rates for selected Items from the NHES Parent/Guardian Interview 



Parent Interview item 


Number 
attempted 


Number 
completed 


Percent 
completed 


Any regular school 


1,009 


1,009 


100.0 


Any care/preschool 


1,498 


1,498 


100.0 


Number of care/preschool 


758 


758 


100.0 


Days per week attends care/preschool 


912^ 


907 


99.4 


Instructional program in care/preschool 


85r 


842 


98.9 


Mother at work during care/preschool 


890^ 


878 


98.6 


Handicapping condition 


1,530 


1,525 


99.7 


Handicap limits access 


72 


72 


100.0 


^^urruiuiy rcjocivmg special eo. services 


72 


71 


98.6 


Read stories with child 


1,530 


1,519 


99.3 


Mother employed 


1.501 


1,475 


98.3 


Father employed 


1,244 


1,232 


99.0 



'Each care/preschool arrangement for each child was asked about separately. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey Field Test. 
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Table 13. -Length of time (in minutes) for interviews, by inclusion in timing analysis summary by interview type 



Interview 


All cases with 
timing information^ 


Cases included in timing 
analysis (excludes outliers) 


Number 


Mean time 


Number 


Mean time 


Screener 


14,933 


4.1 


14,802 


4.0 


Household Respondent Interview 


4,295 


2.0 


3.814 


1.9 


Youth Interview 


1,489 


4.2 


1,262 


3.8 


Parent/Guardian Interview 


1,475 


9.1 


1,271 


8.5 



includes all cases that had a time recorded for a segment without editing. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 1 989 National Household Education 
Survey (NHES) Field Tes'. 




exclusion of the cases with invalid times has resulted in 
large underestimates of interview timings. 

Table 14 presents the length of the interviews 
in minutes by type of instrument. The times are given 
only for interviews that are considered complete (i.e., 
the respondent completed the most critical items). 
These include items concerned with school enrollment 
for the current and previous school years for 14- to 21- 
year-olds, and items concerning participation in child 
care and preschool arrangements for 3- to 5-year-olds. 
As noted previously, the vast majority of the 
respondents completed all of the items for an interview. 
Because of skip patterns in the interviews, all of the 
items were not asked of each respondent. 

The first two sections of the table show the 
length of time for entire interviews and for certain 
combinations of interviews. The Screener took a mean 
of 4.0 minutes to complete per household; the 
Household Respondent Interview 1.9 minutes per 14- to 
21 -year-old; the Youth Interview, 3.8 minutes per 14- 
to 21-year-old; and the Parent/Guardian Interview, 8.5 
minutes to complete per 3- to 5-year-old. 

These overall averages for an interview are less 
informative than the timings for the different 
components of the interviews because of the presence 
of skip patterns. 

The 3.8 minutes for the Youth Interview is an 
underestimate because if the youth was also the 
household respondent, the set of identical items which 
appear in both interviews was not repeated. Rather, the 
CATI system was programmed to apply the sampling 
algorithm and continue with the Youth Interview if the 
person was sampled. The identical items were later 
copied into the appropriate variables. The timings for 
the identical items were only recorded with the HRI 
since they were, in effect, asked only in that interview. 

The second section of the table deals with 
combined interviews (i.e., situations in which more 
than one interview was completed for a household). 
The times in this section were computed only if all of 
the various interviews were completed for persons in 
the household, since including households with any unit 
nonresponses would underestimate the total household 
response time for the study. Comparing the means 
reported in this section with the means computed by 
aggregating the separate interviews indicates that the 
aggregates are reliable predictors of the interview times 
for the combined interview. Therefore, the length of 



time to complete other combinations of interviews can 
be estimated by simply adding up the separate 
components.^ Therefore, the time associated with 
eligible households could be used to form aggregates 
for combinations of interviews. 

The total interview times indicate that the Field 
Test was relatively successful in obtaining the required 
data without overburdening the household respondent. 
The evidence of the timings, especially when combined 
with the results of the item response analysis, supports 
the conclusion that the respondents did not react 
adversely to the interview length. 



Screener Length 

The mean length of time required to complete 
the Screener was 4.0 minutes. The length of the 
Screener was different depending on whether the 
household was eligible or ineligible. Screener 
interviews in households with members who were 
eligible for NHES requiicd the completion of the full 
household enumeration and the acquisition of other 
household characteristics. If the household had no 
eligible members as determined by the enumeration, the 
interview was terminateJ before the other information 
was collected. In addition, ineligible households were 
probably smaller than eligible households (by definition 
ineligible households have no 2- to 6 -year-olds and no 
13- to 22-year-olds) and, therefore, took less time to 
enumerate. 

An operational explanation for some of the 
difference in Screener timings between the eligible and 
ineligible households is related to the introduction of a 
series of eligibility screening items for refusal 
conversion purposes. Several studies (see Fitti,^' for 
example) have suggested that household enumeration in 
the beginning of a survey increases nonresponse. 
Therefore, screening items were used in the refusal 
conversion process to try to convert initial refusals into 
responses. If the household did not meet any of the 
eligibility requirements as identified in the new 
screening items, then the Screener was ended without 
a full household enumeration. If the household 
contained any eligible members, the full enumeration 
was attempted. The household enumerations were not 
completed for about 15 percent of all ineligible 
households as a result of the use of the screening items 
for refusal conversion. 
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BEST COPY AVARABLE 





Interview length in minutes 


Completed instruments 






Standard 


Quartiles 


Number 


Mean 


deviation 


75% 


Median 


25% 


Interview totals: 














Screener 

Household Respondent Interview 
Youth Interview ^ 
rareni inicrvicw 


14,802 
3,814 
1,262 

1 O "7 1 
1 ,Z / 1 


4.0 
1.9 
3.8 
o . o 


1.8 
0.7 
1.3 

A 9 


5.1 

2.3 
4.5 
11.4 


3.7 
1.7 
3.7 
7.2 


2.6 
1.3 
3.0 
5.2 


uomDinBQ interviews. 














Screener/Household Respondent 
Screener/Household Respondent/Youth 
ocrecncr/rdici u 


2,697 
781 
903 


7.7 
1 1.2 
14.0 


1.6 
2.0 
4.6 


8.7 
12.5 
16.8 


7.5 
11.1 
13.1 


6.6 
9.8 
10.3 


Interview components: 














Screener 
Eligible household 
Ineligible household 


5,308 
9,494 


5.5 

3.1 


1.4 
1 .3 


6.3 
3.7 


5.3 
2.8 


4.6 
2.2 


Household Respondent Interview 
Enrollment status last week 
Enrollment status last year 
Earned diploma/certificate 
Handicapping condition 
Employment 
Background 


2,975 
1.387 
1,932 
3,754 
3,527 
3,527 


0.5 

0.6 
0.3 
0.3 
0.5 
0.4 


0.2 
0.3 
0.2 
0.2 
0.3 
0.2 


0.6 
0.7 
0.4 
0.3 
0.6 
0.5 


0.5 
0.5 
0.3 
0.2 
0.4 
0.4 


0.4 
0.4 
0.2 
0.2 
0.3 
0.3 


Youth Interview ^ 
Enrollment status last week 
Enrolment status last year 
Other educational experience 
Earned diploma/certificate 
Handicapping condition 
Employment 
Background 
Educational plans 


421 
1,072 
906 
968 
1,150 
1,150 


U . D 

0.8 
0.7 
0.2 
0.2 
0.6 
2.0 
1.4 


0.4 
0.3 
0.1 
0.1 
0.3 
0.6 
0.7 


0.6 
1.0 
0.8 
0.3 
0.2 
0.7 
9 A 
1.7 


0.4 
0.7 
0.6 
0.2 
0.2 
0.5 
1 .8 
1.2 


0.4 
0.5 
0.5 
0.2 
0.2 
0.3 
1 .5 
0.9 


Parent/Guardian Interview ^ 
Respondent verification/relationship 
Enrolled in firstgrade/kindergarten 
Attends day care/preschool 
Care/preschool arrangement data 
Handicapping condition 
Home environment 


1,271 
951 

1.271 
603 

1,271 

1,271 


1.2 
0.6 
0.6 
6 7 
0.1 
3.0 


0.5 
0.3 
0.3 
2.6 
0.0 
0.8 


1.4 
0.9 
0.8 
7.9 
0.1 
3.4 


1.1 
0.6 
0.6 
6.0 
0.1 
2.9 


0.9 
0.4 
0.4 
5.0 
0.1 
2.4 



^Total time reported for the Youth Interview may be an underestimate, since timings for the first two components 
of the interview are missing for some youths who responded to their own HRl. About 477 youth were their own 
HRl. 

^Times reported for combinations of interviews within households are based only on those households for which alt 
eligible interviews were completed. 

'^The interview lengths are for each eligible 14- to 21-year-old and 3- to 5-year-old in a household, not aggregated 
by household. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, "1989 National Household 
Education Survey Field Test." 
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Based on 5,308 eligible, completed Screeners, 
the mean for the eligible housdiolds was 5.5 minutes 
(see table 13). Three-fourths of these interviews took 
between 4,6 and 6.3 minutes to complete. The eligible 
households are the only ones which were subject to 
additional interviews. 

The mean for the ineligible households was 
considerably smaller; based on 9,494 completed 
ineligible household Screeners it took only 3.1 minutes 
to complete the interview. The length of time required 
for the ineligible households at the 75th percentile is 
only 3.7 minutes, nearly a minute less than the length 
of time required at the 25th percentile for the eligible 
households. 



Household Respondent Interview Length 

The HRl \v^s divided into six components, 
each containing a seriet^ of items about a particular topic 
(see figure 4). Each component took an average of 
approximately 0.5 minutes to complete, with relatively 
little variation from interview to interview. The 
number of interviews varies across the components of 
the HRI because of skip patterns. For example, youths 
were not asked the items about a diploma or certificate 
if they were still enrolled in high school. The 
proportion of interviews for which the times for items 
concerning enrollment status last year were recorded is 
relatively small because of an error in setting the timing 
variable in the CAT! program. The error was 
corrected during the interview period, but the timings 
for many interviews were not retrievable. 



Youth Interview Length 

The Youth Interview was divided into 
components for timing in the same mamier as the HRI 
(see figure 5). The two components that took the 
longest to complete concerned educational plans and the 
youth's background. Since the component on 
educational plans was only asked for dropouts, the 
timing is based on a small number of interviews. As 
noted before, if the youth was the household 
respondent, the identical items (enrollment status last 
week and last year) in the interviews were not repeated. 
The number of interviews for the enrollment status 
items are reduced as a result of this operation. The 
timings for these components in the Youth Interview are 



approximately equal to the timings for the components 
in the HRI, 



Parent/Guardian Interview Length 

The length of the Parent/Guardian Interview is 
very dependent upon the number of care/preschool 
arrangements that were reported for a child (table 15). 
Only about half of tlie sampled 3- to 5-year-olds were 
involved in any care/preschool arrangement, excluding 
kindergarten and first-grade enrollment. For children 
with no care/preschool arrangements, the interview on 
average took about one-third of the time required for a 
child with two arrangements, (As with the other two 
interviews, a flow diagram for the Parent/Guardian 
Interview is shown in figure 6), 

The variability in the t'me required to complete 
the interview was a function of the time required to 
complete the care/preschool arrangement component. 
If a child was not in a care/preschool arrangement, this 
component was skipped. For a child with one, two, or 
three arrangements, the average times required to 
complete this component were 5.8 minutes, 10.6 
minutes, and 13,2 minutes, respectively (not shown in 
table). This factor accounts for the large standard 
deviation and the large interquartile range shown in 
table 13. 

The other relatively time-consuming component 
in the Parent/Guardian Interview was the set of items 
related to the home environment. This component 
includes items on topics such as materials that are in the 
home, television viewing habits and the frequency of 
reading to the child. The mean time for this component 
was 3.0 minutes; the interquartile range, 1.0 minute. 

The evidence from the item response analysis and the 
timings of the Parent/Guardian Interview suggests that 
the burden on the respondent was not so great as to 
introduce significant nonresponse problems. This 
finding appears to be true even for interviews in which 
the parent reported up to three different care/preschool 
arrangements for a child. 



Summary and Conclusions 

The National Household Education Survey is 
an important new data collection program being 
developed by the National Center for Education 
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Figure 4. - Household Respondent Interview (30 - 600 seconds) 



Enrollment status last week 
(5-100 seconds) 



Enrollment status 1 year ago 
(5-100 seconds) 



not enrolled 
el/sec 




enrolled 
e I/sec 



Diploma status 
(5-100 seconds) 



Handicapped condition/ 
services 
(5-100 seconds) 



Employment status 
(5 - 1 00 seconds) 



Background 
(5 - 1 00 seconds) 



Allowed timing ranges for each component are given in parentheses. 
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Figure 5. - Youth interview (30 - 600 seconds) 



Enrollment status last week 
(5-100 seconds) 



$_ 

Enrollment status 1 year ago 
(5-120 seconds) 



enrolled 
el/sec 



Educational experiences: 
grades, 
repeat grade, 
suspended, 
expelled 
(5-100 seconds) 



— J — 




Handicapped condition/ 
services 








(5-100 seconds) 


) 




r 


Employment status 
(5-100 seconds) 








) 






Background 
(5 - 220 seconds) 




K 




J 



not enrolled 
ei/sec 



non- * 
dropout 



Diploma status 
(5 - 1 00 seconds) 




dropout 



Reason for leaving school, 
encouraged to return, 

plan to return, 
plan to get diploma/GED 
(5 - 270 seconds) 



Allowed timing ranges for each component are given in parentheses. 

38 

^6 



Table 15. -Mean of time (in minutes) for Parent interviews, by number of care arrangements 



Number of Arrangements 


Number 


Mean time 


None 




665 


5.3 


One 




492 


11.1 


Two 




109 


15.9 


Three 




5 


18.7 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1989 National Household 
Education Survey (NHES) Field Test. 
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Figure 6. - Parent/Guardian Interview (30 - 1500 seconds) 



no 



Confimi child's age, 
collect household members' 
relationships to child 
(5 - 200 seconds) 



4- and 5- year-olds 



Kindergarten or first grade 
enrollment, public/private, 
extended day programs 
(5-100 seconds) 




Regular care or preschool 
arrangements? 
(5-100 seconds) 



yes 



For each arrangement: 
care provider, place, hours, cost, # 
children & staff, educational content, 

activities, reasons for selecting 
care/preschool, parent's activiites while 
child is at arrangement 
(100 - 1000 seconds) 



I 



Does child have handicap; nature 
of handicap, services received, 
services before age 2 
(1-20 seconds) 



I 



Educational environnment in the 
home: educational 
materials, activities, reading, 
television, parent's educ. 
(15-30 seconds) 



3-year-olds 
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Statistics. The NHES is a telephone survey program 
that can be used to obtain detailed information on a 
variety of education issues. It offers the potential for 
examining education issues that have been difficult to 
address through the more traditional NCES surveys that 
are based on institutional samples. 

In the fall of 1989 a field test of the NHES 
was implemented to study two subject matter areas and 
a host of methodological issues. The two subject areas 
were high school dropouts and the early childhood care 
and education of children. The key methodological 
issues were oversampling black and Hispanic telephone 
clusters, multiplicity sampling to identify additional 
dropouts, assessing undercoverage bias, and comparing 
youth and household proxy responses concerning 
education status. Each of these topics is presented in 
detail in the other reports in this series, cited earlier in 
this report. 



The telephone methodology of the NHES 
offers some important advantages for the program. 
One of the major advantages of a telephone survey of 
households is that it is much less expensive than a 
comparable personal interview survey. It can also be 
instituted relatively quickly, address several subject 
matter areas at the same time, and provide results in a 
very timely fashion. 

The results of the Field Test were encouraging 
for the future of the NHES. The Field Test showed 
that many of the concerns could be adequately handled 
in practice. It also revealed that each topic has its own 
unique set of circumstances that must be explored 
individually. Some topic areas will be more 
appropriate than others for the approach used in the 
NHES. For those topics which are not well suited for 
the NHES methodology, special studies or tests may be 
needed before implementation. 
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